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(57) Abstract 

"^general purpose, programmable media processor (12) for processing and transmitting a media data streams. The media processor 
(12) incorporates an execution unit (100) that maintains substantially peak data throughout of media data streams. The execution unit 
(100) deludes a dynamically partionable multi-precision arithmetic unit (102), programmable switch (104) and programmable extended 
mathematical element (106). A high bandwidth external interface (124) supplies media data streams at substantially peak rates to a general 
purpose register file (110) and the execution unit. A memory management unit, and instruction and data cache/buffers (118, 120). The 
general purpose, programmable media processor (12) is disposed in a network fabric consisting of fiber optic cable, coaxial cable and 
twisted pair wires to transmit, process and receive single or unified media data streams. 
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WE CLAIM: 

1 - An execution unit that maintains substantially peak data throughput 
in the unified execution of multiple media data streams, the execution unit having 
a data path, comprising: 

a multi-precision arithmetic unit coupled to the data path, the multi- 
precision arithmetic unit capable of dynamic partitioning based on the elemental 
width of data received from the data path; 

a switch coupled to the data path and programmable to manipulate 
data received from the data path, the switch providing data streams to the data 
path; and 

an extended mathematical element coupled to the data path and 
programmable to implement additional mathematical operations at substantially 
peak data throughput. 

2. The execution unit defined in claim 1 , wherein the multi-precision 
execution unit is configurable to divide the data into component symbols of various 
sizes, analyze the component symbols based upon instructions, and re-synthesize 
the component symbols for communication over the data path. 

3. The execution unit defined in claim 2. wherein the multi-precision 
execution unit is operable to perform unique operations on each component 
symbol. 



4. The execution unit defined in claim 2. wherein the mathematical 
element is operable to perform finite group, finite field, finite ring and table look- 
up operations on the symbols. 

5. The execution unit defined in claim 1, wherein the arithmetic unit is 
programmable to perform Boolean, integer and floating point mathematical 
operations. 
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6. The execution unit defined in claim 5, wherein the operations 
performed by the arithmetic unit are capable of being performed at various levels 
of precision. 

7. The execution unit defined in claim 1, wherein the manipulation of 
data comprises copying, shifting and re-sizing data. 

8. The execution unit defined in claim 1, further comprising control to 
maximize use of the execution unit by performing operations at peak data width of 
the data path. 

9. The execution unit defined in claim 2, wherein the size of 
component symbols match. 

10. An execution unit having a data path, comprising: 
at least one register file coupled to the data path; 

a multi-precision arithmetic unit coupled to the data path, the multi- 
precision arithmetic unit capable of dynamic partitioning based on the elemental 
width of data received from the data path; 

a switch coupled to the data path and programmable to manipulate 
data received from the data path, the switch providing data streams to the data 
path; and 

an extended mathematical element coupled to the data path and 
programmable to implement additional mathematical operations at substantially 
peak data throughput. 

11. An execution unit having a data path, comprising: 

a multi-precision arithmetic unit coupled to the data path, the multi- 
precision arithmetic unit capable of dynamic partitioning based on the elemental 
width of data received from the data path; 
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means coupled to the data path for manipulating data received from 
the data path, the means for manipulating data being programmable and providing 
a data signal to the data path; and 

an extended mathematical element coupled to the data path and 
programmable to implement additional mathematical operations at substantially 
peak data throughput. 

12. A general purpose programmable media processor having an 
instruction path and a data path to digitally process a plurality of media data 
streams, comprising: 

a high bandwidth external interface operable to receive a plurality of 
data of various sizes from an external source and communicate the received data 
over the data path at a rate that maintains substantially peak operation of the media 
processor; 

at least one register file configurable to receive and store data from 
the data path and to communicate the stored data to the data path; and 

a multi-precision execution unit coupled to the data path, the multi- 
precision execution unit configurable to partition data received from the data path 
to account for the elemental symbol size of the plurality of media data streams, 
and programmable to operate on the data to generate- a unified symbol output to 
the data path. 

13. The media processor defined in claim 12, wherein the execution unit 
is dynamically configurable to partition data received from the data path. 

14. The media processor defined in claim 12, further comprising: 
means for moving data between registers and memory by 

performing load and store operations, and for coordinating the sharing of data 
among a plurality of tasks by performing synchronization operations based upon 
instructions and data received by the execution unit; 



409 



WO 97/07450 



PCT/US96/13047 



means for securely controlling the sequence of execution by 
performing branch and gateway operations based upon instructions and data 
received by the execution unit; and 

a memory management unit, the memory management unit operable 
to retrieve data and instructions for timely and secure communication over the data 
path and instruction path. 



15. The media processor defined in claim 14, further comprising: 

a combined instruction cache and buffer, the combined instruction 
cache and buffer dynamically allocated between cache space and buffer space to 
ensure real-time execution of multiple media instruction streams; and 

a combined data cache and buffer, the combined data cache and 
buffer dynamically allocated between cache space and buffer space to ensure real- 
time response for multiple media data streams. 

16. The media processor defined in claim 15, wherein real-time 
execution is ensured by dynamically allocating instruction buffer space to the 
smallest and most frequently executed blocks of media instructions. 

17. The media processor defined in claim 15, wherein real-time 
response is ensured by dynamically allocating data buffer space to the smallest and 
most frequently accessed working sets of media data. 

18. The media processor defined in claim 12, wherein media data 
streams comprise Nyquist sampled inputs and outputs. 

19. The media processor defined in claim 12, wherein media data 
streams originate from standard computer memory and I/O interfaces. 

20. The media processor defined in claim 12, wherein the multi- 
precision execution unit is configurable to divide the data into component symbols 
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of various sizes, analyze the component symbols based Upon instructions, and re- 
synthesize the component symbols for communication over the data path. 

21 . The media processor defined in claim 12. wherein the plurality of 
media data streams comprise presentation media information, transmission media 
information, and storage media information. 

22. The media processor defined in claim 2 1 , wherein presentation 
media information comprises audio, video, image, and graphical information 

23. The media processor defined in claim 21. wherein transmission 
media information comprises radio and network data transmissions;. 



24. 

information comprises 



The media processor defined in claim 21, wherein storage media 
•mprises data encoded in moving and solid-state memory media. 

25. The media processor defined in claim 12, wherein the width of the 
data path is at least 128 bits. 

26. The media processor defined in claim 12, wherein the multi- 
precision execution unit comprises a dynamically partitionable arithmetic unit, a 
register controllable cross-bar switch, and an extended mathematical element. 

27. The media processor defined in claim 24, wherein the register 
controllable cross-bar switch comprises a Benes network design. 

28. The media processor defined in claim 26, wherein the register 
controllable cross-bar switch is programmable and is operable to manipulate 
symbols. 
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29. The media processor defined in claim 22, wherein the extended 
mathematical element is operable to perform finite group, finite field, finite ring 
and table look-up operations on the symbols. 

30. The media processor defined in claim 12, further comprising a set 
of predefined instructions accessible by a user. 

31. The media processor defined in claim 13, wherein the means for 
performing load, store, and synchronization operations and the means for 
performing branch and gateway operations comprises a set of predefined 
instructions accessible by a user. 

32. The media processor defined in claim 31 . wherein the predefined 
instructions are combinable to implement composite functions on the plurality of 

media data streams. 

33. A high bandwidth processor interface for receiving and transmitting 
a media stream, comprising: 

a data path, the data path operable to transmit media information at 
sustained peak rates; 

a plurality of memory controllers, the plurality of memory 
controllers coupled to the data path in series to communicate stored media 
information to and from the data path; and 

a plurality of memory elements coupled to each of the plurality of 
memory controllers in parallel, the plurality of memory elements for storing and 
retrieving the media information. 

34. The high bandwidth processor interface defined in claim 33 
wherein the data path comprises a plurality of data paths forming a high bandwidth 
data channel. 
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35. The high bandwidth processor interface defined in claim 34 
wherein the high bandwidth data channel is unidirectional. 

36. The high bandwidth processor interface defined in claim 33 further 
compnsing a general purpose programmable media processor coupled to the high 
bandwidth data channel to receive, process and transmit media information at 
substantially peak rates. 

37. The high bandwidth processor interface defined in claim 33 
wherein the peak rate of operation comprises at least one gigabyte of information 
per second from point to point. 

38. The high bandwidth processor interface defined in claim 33 
wherein the plurality of memory controllers each comprise a paired link disposed 
between each memory controller, the paired links each for transmitting and 
receiving plural bits of data and having differential data inputs and outputs and a 
differential clock signal. 

39. The high bandwidth processor interface defined in claim 38 
wherein the paired link further comprises a digital skew calibrator to adjust the 
plural bits of data relative to the differential clock signal to eliminate skew 
between the data. 

40. The high bandwidth processor interface defined in claim 38 
wherein the paired link further comprises a phase locked loop to eliminate jitter in 
the differential clock signal transmitted between paired links. 

41 . The high bandwidth processor interface defined in claim 38, 
wherein the plural bits comprise eight bits of data. 
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42. The high bandwidth processor interface defined in claim 38 
wherein the paired links each further comprise termination resistors to form 
matched impedances for each paired link. 

43. The high bandwidth processor interface defined in claim 34 
wherein the high bandwidth data channel comprises plural parallel high bandwidth 
data channels. 

44. A system for unified media processing comprising: 

a plurality of general purpose media processors, each media 
processor operable at sustained peak data rates and having a dynamically 
partitioned execution unit and a high bandwidth interface, the high bandwidth 
interface coupled to external memory and input/output elements to receive and 
transmit data to the media processor at substantially peak rates; 

a bidirectional communication fabric, the plurality of media 
processors coupled to the bi-directional communication fabric to transmit and 
receive at least one media stream comprising presentation, transmission, and 
storage media information. 

45. The system defined in claim 44, wherein the bi-directional 
communication fabric comprises a fiber optic network. 

46. The system defined in claim 44, wherein the bi-directional 
communication fabric comprises an heterogeneous network. 

47. The system defined in claim 44. wherein the bi-directional 
communication fabric comprises a coaxial cable network. 

48. The system defined in claim 44, wherein the bi-directional 
communication fabric comprises a wireless network. 
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49. The system defined i„ claim 44. wherein a subset of the plurality of 
media processors comprise network servers. 

50. The system defined in claim 44, wherein the plurality of media 
processors are programmable by downloading program information over the bi- 
directional communication fabric. 



51. The system defined in claim 44, wherein the each of the plurality of 
medxa processors can access an idle execution unit of another media processor in a 
shared manner to efficient* process presentation, transmission and storage media 
information at substantially peak data rates. 

52. The system defined in claim 44, wherein each media processor 
further comprises dedicated memory and wherein the each of the plurality of 
media processors can employ any unused portion of the dedicated memory of 
another media processor in a shared manner to efficiently store and retrieve 
presentation, transmission and storage media information at substantially peak data 
rates. 

53. A parallel multi-processor system that maintains substantially peak 
data throughput in the unified execution of multiple media streams, the system 
having a data path, comprising: 

at least one high bandwidth external interface, the at least one high 
bandwidth external interface coupled to the data path and operable to receive a 
plurality of data of various sizes from an external source and communicate the 
received data at a rate that maintains substantially peak operation of the parallel 
multi-processor system; 

a plurality of register files, each register file having at least one 
general purpose register coupled to the data path and operable to store a working 
set of media data; and 

at least one multi-precision execution unit coupled to the data path 
the at least one multi-precision execution unit dynamically configurable to partition 



415 



WO 97/07450 



PCT/US96/13047 



data within a working set of media data received from the data path to account for 
the elemental symbol size of the plurality of media streams, and programmable to 
operate in parallel on working sets of data stored in the plurality of register files to 
generate a unified symbol output for each register file. 

54. The parallel multi-processor system defined in claim 53. wherein 
the at least one execution unit alternates in a round robin manner to operate on 
data stored in the plurality of register files. 

55. The parallel multi-processor system defined in claim 53. further 
comprising an instruction pre-fetch pipeline. 

56. The parallel multi-processor system defined in claim 55, wherein 
the instruction pre-fetch pipeline comprises a super-string pipeline. 

57. The parallel multi-processor system defined in claim 55, wherein 
the instruction pre-fetch pipeline comprises a super-spring pipeline. 

58. The parallel multi-processor system defined in claim 53. further 
comprising a data pre-fetch pipeline. 

59. The parallel multi-processor system defined in claim 58. wherein 
the data pre-fetch pipeline comprises a super-string pipeline. 

60. The parallel multi-processor system defined in claim 58, wherein 
the data pre-fetch pipeline comprises a super-spring pipeline. 

61. The parallel multi-processor system defined in claim 53, further 
comprising a requester, responder and transponder daemon. 

62. A method for processing unified streams of media data, comprising 
the steps of: 
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receiving a stream of unified media data including presentation, 
transmission and storage information; 

dynamically partitioning the unified stream of media data into 
component fields of at least one bit based on the elemental symbol size of data 
received; and 

processing the unified stream of media data at substantially peak 

operation. 



63. The method defined in claim 62, wherein the step of processing the 
unified stream of media data comprises the steps of: 

storing the stream of unified media data in a general register file; 

performing multi-precision arithmetic operations on the stored 
stream of unified media data based on programmed instructions, the multi- 
precision arithmetic operations including Boolean, integer and floating point 
mathematical operations; 

manipulating the component fields of unified media data based on 
programmed instructions that implement copying, shifting and re-sizing operations; 
and 

performing multi-precision mathematical operations on the stored 
stream of unified media data based on programmed instructions, the mathematical 
operations including finite group, finite field, finite ring and table look-up 
operations. 

64. The method defined in claim 63, further comprising the steps of: 
pre-fetching instructions and data to fill instruction and data 

pipelines; 

performing memory management operations to retrieve instructions 
and data from external memory; 

storing instructions and data in instruction and data cache/buffers; 

and 

dynamically allocating buffer storage in the instruction and data 
cache/buffers to ensure real-time execution. 
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65. The method defined in claim 63. further comprising the step of 
providing a set of instructions to process the stream of unified media data, the set 
of instructions including load, store, synchronization, branch and gateway 
instructions. 

66. The method defined in claim 65, further comprising the step of 
programming a sequence of at least one instruction from the set of instructions. 

67. A method for achieving high bandwidth communications between a 
general purpose media processor and external devices, comprising the steps of: 

providing a high bandwidth interface disposed between the media 
processor and the external devices, the high bandwidth interface comprising at 
least one uni-directional channel pair having an input port and an output port; and 

transmitting and receiving a plurality of media data streams, 
comprising component fields of various sizes between the media processor and the 
external devices at a rate that sustains substantially peak data throughput at the 
media processor. 

68. The method defined in claim 67. wherein the step of providing a 
high bandwidth interface further comprises providing a plurality of external 
devices, the plurality of external devices coupled in series on the at least one uni- 
directional channel pair. 

69. The method defined in claim 67, wherein the step of providing a 
high bandwidth interface further comprises providing a plurality of parallel uni- 
directional channel pairs. 



70. A method for processing streams of media data, comprising the 
steps of: 

providing a bi-directional communications fabric for transmitting 
and receiving at least one stream of unified media data, the at least one stream of 
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unified media data comprising presentation, transmission and storage information; 
and 

providing at least one programmable media processor within the 
communications network, the at least one programmable media processor for 
receiving, processing and transmitting the at least one stream of unified media data 
over the bi-directional communications fabric. 
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